perf(relay): index channel-id lookups and skip trace-only reads - #4647

Problem

SELECT id, community_id FROM channels WHERE id = ANY($1) AND deleted_at IS NULL is the top Load by waits (AAS) on the Buzz Postgres writer. Two independent causes compound, and both are fixed here.

1. No index can serve it

channels is PRIMARY KEY (community_id, id), and every secondary index leads with community_id:

The two tenant-independent lookups carry no community_id predicate — deliberately:

• Db::communities_of_channels — WHERE id = ANY($1) AND deleted_at IS NULL
• Db::community_of_channel — WHERE id = $1 AND deleted_at IS NULL

That independence is load-bearing, not an oversight: projecting a row's true owning community regardless of the fetch query's WHERE clause is what makes Inv_NonInterference non-vacuous. If the fetch ever dropped its tenant scoping, this lookup would still report the real label and the checker would catch the mismatch.

But a composite btree is only usable when its leading column is constrained, so neither query can use the primary key, and nothing else leads with id. Both sequentially scan channels on every call.

2. In production the result is discarded

Both call sites feed record_read_message_rows / record_read_by_id_rows, which call tracer.record(...). Production binds NoopTracer (crates/buzz-relay/src/state.rs), whose record body is empty.

The existing guard tests trace_state, which is Some for every well-formed request — it only goes None on malformed pubkey bytes. So the scan ran on the hot read path and its output was dropped. This is the classic eager-argument bug: log.debug("..." + expensiveCall()) with no isDebugEnabled() check.

3. Multiplied per filter

The non-search call site sits inside the phase-3 per-filter loop, so a REQ carrying N filters performed N sequential scans of channels before responding.

Changes

Tracer::enabled() — a capability check on the trait (the isDebugEnabled() of this seam), defaulting to true. NoopTracer overrides it to false, and both emitters in req.rs now gate on it, skipping the trace-only DB read entirely in production.

migrations/0027_channels_id_lookup_index.sql

CREATE INDEX IF NOT EXISTS idx_channels_id_live
    ON channels (id) INCLUDE (community_id)
    WHERE deleted_at IS NULL;

• INCLUDE (community_id) — both queries select exactly (id, community_id), so this is covering and can be served index-only.
• Partial on deleted_at IS NULL — matches both predicates exactly, excludes soft-deleted history, and lets Postgres skip the recheck.
• Not UNIQUE. id alone is not unique in this table — command_executor.rs documents that community_of_channel(channel_id) is ambiguous because the same channel id can appear under more than one community. A unique index would encode a false constraint and fail to build on any database already holding such a pair.

Worth keeping the index even though fix #1 removes the production caller: it still runs under conformance, and community_of_channel has the same problem on its own paths.

schema/schema.sql — mirrored, since a test asserts desired-state parity.

Conformance is unchanged

This is the part worth reviewing closely. Under a real tracer enabled() returns true and every emit happens exactly as before — the gate only skips building emit inputs when nothing observes them, never an emit that would otherwise have been made. The coverage-breach guard stays non-vacuous.

CountingTracer forwards enabled() to its inner tracer rather than inheriting the true default. Both directions matter and both fail silently:

• inheriting true over a NoopTracer would keep the overhead this PR removes;
• hardcoding false over a live tracer would suppress the emits whose absence EmitGuard reports as ImplBug — masking real breaches behind expected ones.

Covered by a new regression test, counting_tracer_delegates_enabled_to_inner, which asserts delegation in both directions.

Verification

• cargo check -p buzz-conformance -p buzz-relay — clean
• cargo clippy --all-targets — clean, zero warnings
• cargo test -p buzz-conformance — 6/6
• cargo test -p buzz-relay --lib conformance — 11/11
• cargo test -p buzz-db --lib migration — 7/7
• just test-unit (pre-push) — green

Migration-count assertions in crates/buzz-db/src/migration.rs were bumped 26 → 27, with content assertions for 0027 following the existing per-migration pattern (including a guard that it never becomes UNIQUE).

Open questions for reviewers

1. Lock strategy. Built without CONCURRENTLY, following migration 0004's precedent, because sqlx runs each migration inside a transaction and CREATE INDEX CONCURRENTLY cannot run in one. This takes a brief SHARE lock on channels (blocks writes, not reads) — small relative to events, but an operator preferring zero write-blocking can pre-build it by hand and IF NOT EXISTS makes the migration a no-op. I could not confirm whether sqlx 0.9 supports a -- no-transaction directive; if it does, that may be preferable.

2. Diagnosis is static. This comes from reading the source, not from EXPLAIN against the live database. Worth confirming with EXPLAIN (ANALYZE, BUFFERS) on the writer before/after — that also sizes the win by revealing the real table size and row counts.

3. Expected impact scales with average filters-per-REQ, which I did not measure. pg_stat_statements ordered by total_exec_time would confirm this query drops off the top and show whether anything else is scanning the same way.